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Evidence for 
Excellence in 
Education 





What you don’t find out about 



England’s educational performance 


in the PISA league table 


• It isn’t always possible to say with certainty 
from looking at a country’s rank in the PISA 
educational league tables alone whether one 
country or economy has definitely performed 
better than another. 

• England’s position in the league tables 
is dependent on which countries and 
economies participate in a given year and 
whether the performance of these countries 
and economies is better, worse or the same 
as in the last round of PISA. 


• England has maintained the same level of 
performance in maths in the last three rounds 
of PISA (2006, 2009, 2012). England’s relative 
position compared to the number of higher 
performing countries and economies has 
remained relatively stable over time, although 
some countries have overtaken England and 
are now performing significantly better. 

• PISA data alone doesn’t tell us why some 
countries and economies are higher 
achieving, why some pupils perform better 
than others or which teaching practices 
result in higher performance. 


England’s position in the league tables is dependent 
on which countries and economies participate in 
a given year 
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The is a survey of the educational 

achievement of 15-year-olds developed by the Organisation for Economic Co-operation and 
Development (OECD). PISA assesses students’ maths, science and reading skills. 15-year- 
olds from across the world take tests in the three key subjects, with a focus on one subject 
in each year of the assessment. In PISA 2012, the main subject was maths and in the next 
PISA survey in 2015 the focus will be on science. 

All four countries of the UK participated in PISA 2012. This briefing document focuses on 
England. Where relevant the briefing also references the findings for the whole of the UK. 

This piece sets out to explore how PISA results can be used to judge the success of 
education systems, particularly in terms of the claims that are made about PISA findings by 
politicians and the media. There are ongoing debates surrounding the methodology of PISA 
(for example, see this in the TES), which this piece does not seek to do address. 


Here’s what it can tell us ... 


England’s performance has generally stayed the same since 2006 


When the results of PISA 2012 were released, 
the focus from politicians and the media was 
primarily on how the UK ranked against the 
other participating countries and economies. 
Michael Gove, Secretary of State for Education 
at the time, claimed “Our performance in these 
league tables has been at best, stagnant, at 
worst declining. In the latest results we are 
21st amongst 65 participants in the world 
for science, 23rd for reading and 26th for 
mathematics.” 

However, while England (and the UK taken 
as a whole) may have been ranked 26th in 
the world for maths, based on its score there 
were actually only 19 countries and economies 
which significantly outperformed England. 

Each country’s PISA score has a margin of 
error associated with it (because of factors 
like not every pupil in the country is tested, 
and because the pupils who are tested may 
have performed particularly well or badly on 
the day). This means we cannot conclude that 
differences between similar scores (over time 
or across countries) reflect genuine differences 


in performance or have arisen simply due to 
chance. Differences which are large enough 
that they’re unlikely to have arisen solely by 
chance are termed “significant differences”. 

So it isn’t always possible to tell from looking 
at the rankings in the league table whether 
one country or economy has significantly 
performed better than another. 

Looking at whether this score is higher or lower 
than in the previous PISA assessment year also 
doesn’t tell us accurately whether England has 
improved, or is “in decline” or “stagnating”. We 
have to establish whether the score for that 
particular year is significantly different from 
the score obtained previously. For example 
in 2006 England’s PISA score for maths was 
495, in 2009 it was 493 and in 2012 it was 
495. Although the score has fluctuated over 
the three PISA assessment years, analysis by 
NFER found that, in 2012, England maintained 
the same level of performance in maths as in 
the previous two rounds of PISA (in 2009 and 
2006). 


Looking at whether this score is higher or lower than 
in the previous PISA assessment year also doesn’t tell 
us accurately whether England has improved 



2 






election factsheet 


If we want to get a sense of whether England 
is “falling behind global rivals” we can look 
at how many countries and economies 
outperformed England in a particular subject in 
the previous rounds of the PISA assessment. 
The number of countries and economies 
outperforming England in maths was 18 in 
2006, and 20 in 2009-just above the 19 in 
2012 . 

So England’s position in terms of the number 
of countries and economies performing 
significantly better has remained relatively 
stable over time. 

England’s position relative to other countries 
and economies is not just based on our own 
performance. It’s also influenced by which 
countries and economies participate and 
whether their own performance is better, 
worse or the same as in the last round of PISA. 
The significant improvement of some high- 
performing countries and economies since 
PISA 2009, such as Shanghai, Singapore, 
Macao-China and Poland, mean that England 
has fallen further behind these particular 
countries and economies. 

England’s performance relative to its global 
rivals also depends on who you count as 
global rivals. If you wanted to look at England’s 
performance compared to other English- 
speaking economies, only one (Australia) had a 
maths score that was significantly higher than 
England’s in 2012. (Canada also outperformed 
England but it’s not considered to be a solely 
English speaking country as the tests there are 
administered in French too). 


PISA also provides 
data on the “spread” 
of achievement 


PISA scores on the maths scale 

2012 PISA results 


Shanghai-China 

613 



Singapore 

573 



Hong Kong-China 

561 



Chinese Taipei 

560 



South Korea 

554 



Macao-China 

538 



Japan 

536 



Liechtenstein 

535 



Switzerland 

531 



Netherlands* 

523 



Estonia* 

521 



Finland* 

519 



Canada 

518 



Poland* 

518 



Belgium* 

515 



Germany* 

514 



Vietnam 

511 

Significantly 

Austria* 

506 

higher than 

Australia 

504 

England 

Republic of Ireland* 

501 

A 

L 

Slovenia* 

501 



Denmark* 

500 



New Zealand 

500 



Czech Republic* 

499 



Scotland 

498 



England 

495 

No 

France* 

495 

significant 

OECD Average 

494 

difference 

Iceland 

493 



Latvia* 

491 



Luxembourg* 

490 



Norway 

489 



Portugal* 

487 



Northern Ireland 

487 

1 

r 

Italy* 

485 

Significantly 

Spain* 

484 

lower 

Russian Federation 

482 



Slovak Republic* 

482 



United States 

481 



Lithuania * 

479 



Sweden* 

478 



Hungary* 

477 



Croatia* 

471 



Wales 

468 



Israel 

466 



Greece* 

453 



Serbia 

449 



Turkey 

448 



Romania* 

445 



Cyprus 

440 



Bulgaria* 

439 



United Arab Emirates 

434 



Kazakhstan 

432 



Chile 

423 



Mexico 

413 

1 

r 


* EU countries, italicised-OECD, non-italicised-non-OECD 
Source: Department for Education Achievement of 15-Year-Olds in 
England, PISA 2012 National Report 
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The difference in performance in maths by low achieving 
and high achieving English students is relatively large 

A country’s average PISA score doesn’t tell us everything someone might need to know 
to judge whether a country has a successful education system. For example, PISA also 
on the “spread” of achievement (the achievement gap). England has a 
relatively wide spread of achievement— that is, although there are a number of students who 
do well in PISA there are many who don’t. 

Five of the highest performing countries and economies have a wider achievement gap 
(Chinese Taipei, Singapore, Shanghai, South Korea and Hong Kong)— although their overall 
performance is better than that seen in England. 


Advantaged pupils are 
about two years ahead of 
disadvantaged pupils 

The OECD finds that, on average, the 
performance difference between advantaged 
and disadvantaged students across the 
countries and economies participating in 
PISA is equivalent to more than two years of 
schooling. England’s PISA 201 2 maths data 
shows England is about average in this respect. 
Three of the top five performing countries and 
economies in maths (Shanghai, Singapore 
and Chinese Taipei) perform worse than the 
average, for example in Chinese Taipei the 
performance difference between advantaged 
and disadvantaged students is equivalent to 
more than three years of schooling. 

The OECD also uses the PISA data to evaluate 
how much socio-economic backgrounds 
determine achievement in each country. DfE 
analysis of the 2009 PISA data found that 
in England the effect of socio-economic 
background on reading attainment was greater 
than that seen on average across OECD 
countries. A similar picture was seen for maths 
attainment in PISA 201 2, in only 12 OECD 
countries was the effect of socio-economic 
background on maths attainment larger than 
that seen in England. 

Compared to the UK, backgrounds have less of 
an impact on performance in some of the high- 
performing countries and economies such as 
Canada, Estonia, Finland and Hong Kong. 


The majority of English 
students report being happy 
and satisfied at school 

PISA 2012 was the first PISA survey that asked 
students to evaluate their happiness at school. 

The PISA data shows that, on average, 80% 
of students indicate that they are happy 
with school and suggests that, according to 
the OECD, the majority of the countries and 
economies that participated in PISA 2012 are 
consequently able to “foster student well- 
being”. 

When NFER analysed the PISA 2012 national 

data, it found that students in England reported 
a similarly high sense of belonging and 
satisfaction with school, with 84% of students 
identifying themselves as happy at school, and 
85% as satisfied with school. 

This is a different picture to that seen in the 
highest achieving Pacific Rim countries and 
economies where, although, a high percentage 
of students in these economies are happy at 
school (82%-89%) they are not necessarily 
as satisfied with their schools (60%-81 %). 

The exception is South Korea where a smaller 
proportion of students reported being happy 
or satisfied with their school (they came lowest 
in the happiness measure with only 60% of 
students reporting they were happy). Without 
further information we do not know if this lower 
level of satisfaction is the result of students in 
the Pacific Rim countries and economies being 
more demanding consumers of education. 
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What PISA doesn’t tell us ... 


The success of East Asian 
countries and economies 
means that their teaching 
practices “make the difference” 

PISA provides us with limited information about 
the teaching practices that are used in high- 
achieving countries and economies as it does 
not collect information about these directly 
from teachers. It only captures this information 
through reports from students about what they 
do in their lessons. 

Other international surveys such as the 
Teaching and Learning International Survey 
(TALIS), TIMSS and PIRLS surveys collect 
information about teaching practices directly 
from teachers. We can use these to look at 
the link between achievement and particular 
teaching practices, but it still can’t tell us 
whether the high performance of economies 
such as Shanghai, Singapore and Hong Kong 
is a result of specific teaching practices. 

PISA can help identify which are the higher 
performing countries, and give us some 
indication of the things that are associated 
with higher performance. However, PISA can’t 
give us a detailed understanding of why some 
countries and economies are higher achieving, 
why some pupils perform better than others 
or which teaching practices result in higher 
performance. 

Why certain students perform 
better than others 

You may have seen this headline: 

I The UK is so far behind Shanghai that 
children of professionals here do worse 
than children of cleaners in Shanghai 

The OECD did find that in the UK students 
whose parents work as professionals do not 
perform as well in maths as either the children 
of professionals in other economies or the 
children of manual workers (e.g. in Shanghai 
and Singapore). 


But when we compare PISA scores for a 
specific occupation across countries and 
economies we are not necessarily comparing 
like with like. For example 33% of students 
in the UK report having parent(s) working in 
professional occupations, while for Shanghai 
only about 1 5% do. The data is also based 
on students’ responses of what their parents’ 
occupations are, so there could be some error 
in the responses. 

Simply looking at occupations does not 
take into account parent’s expectations and 
aspirations for their children-and how this 
is manifested in what they do to encourage 
their children to be high achievers either. For 
example, do they spend time working with 
them on homework assignments; do they pay 
for tutors? 

In some countries and economies, there may 
be greater parental expectations of children’s 
performance; parents may put more pressure 
on their children to work harder; or they may 
invest more household income on private 
education. 

The Nuffield Foundation found that in China 
increased parental expectations were the 
same across different socio-economic 
groups. Therefore, the expectations of parents 
employed in elementary occupations in one 
country may exceed the expectations of 
parents employed in managerial or professional 
occupations in another. We don’t know what 
the case is in England. 

Without looking at other background 
characteristics (many of which are not collected 
through PISA) it is not possible to say why 
students with parents in different occupational 
groups perform differently. 


PISA can help identify 
which are the higher 
performing countries 
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Students’ parental occupation in PISA 

Percentage of students participating in PISA by parental occupation, reported by students in PISA 
and coded by International Standard Classification of Occupations 



UK 

Shanghai 

Managers 

22% 

28% 

Professionals 

33% 

15% 

Technicians and associate professionals 

14% 

26% 

Clerical support workers 

12% 

13% 

Service and sales workers 

32% 

31% 

Skilled agricultural, forestry and fishery workers 

3% 

2% 

Craft and related trades workers 

19% 

14% 

Plant and machine operators and assemblers 

25% 

8% 

Workers in elementary occupations (e.g. cleaners) 

35% 

19% 

Source: OECD PISA 2012 


From this analysis, OECD researchers found 
that, in most countries and economies, the 
children of professionals have, on average, the 
best results in maths. 


This briefing was written by Bethan Burge of the National Foundation 
for Educational Research in collaboration with Full Fact. This work 
has been funded by the Nuffield Foundation but the content is the 
responsibility of the authors and of Full Fact, and not of the Nuffield 
Foundation. 
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